A Comparative Study of Search Result Diversification Methods

Top-ranked documents returned by traditional retrieval functions may cover the same piece of relevant information and cannot satisfy different user needs. Search result diversification solves this problem by diversifying results to cover more information needs, i.e., query subtopics, in top-ranked documents. Many diversification methods have been proposed and studied, and most of them re-rank original retrieved documents according to both relevance and diversity functions in a probabilistic framework. Although official TREC results make it possible to compare the effectiveness of different diversification systems, it remains unclear whether the better performance of a system comes from better diversification methods or component estimation methods. In this paper, we conduct a systematic study on comparing three representative diversification methods which can be implemented using probabilistic methods. We not only analytically compare the methods but also conduct empirical studies and evaluate the effectiveness of these methods in a controlled manner.